On Algorithms for Simple Stochastic Games

We survey a number of algorithms for the simple stochastic game problem, which is to determine the winning probability of a type of stochastic process, where the transitions are partially controlled by two players. We show that four natural approaches to solving the problem are incorrect, and present two new algorithms for the problem. The rst reduces the problem to that of nding a locally optimal solution to a (non-convex) quadratic program with linear constraints. The second extends a technique of Shapley called the successive approximation technique, by using linear programming to maximize the improvement at each approximation step. Finally, we analyze a randomized variant of the Hooman-Karp strategy improvement algorithm.